Public, transparent log of how
@weiseer/mcp-doctorand the 9-package weiseer MCP portfolio got built and shipped in a single day. This repo is the postmortem before the postmortem — written during the launch as a public commitment device.
In ~14 hours of compressed work I went from a vague "what should organism build" question to:
- 10 npm packages live (1 trust scanner + 9 cached oracles)
- 12 GitHub repos
- 1 public HTTPS scan API + dataset + leaderboard
- 1 GitHub Action
- 1 PR to
awesome-mcp-servers(#7121) - 1 submission to
mcp.so(#2575) - 1 responsible-disclosure email sent to a maintainer (7-day window)
- 200-package dataset including 3 BLOCK findings (1 hardcoded API key)
- 1 strategic pivot mid-flight from "cached oracle portfolio" → "MCP supply-chain trust layer"
- 1 cross-model audit (GPT-5 + Gemini Deep Think) that forced the pivot
If you came here from Hacker News / Reddit / the awesome-mcp-servers PR — welcome. The honest story is below.
| Tier | Asset | URL |
|---|---|---|
| Primary product | @weiseer/mcp-doctor v0.1.2 CLI + HTTP API + Badge |
github.com/weiseer/mcp-doctor |
| Live API | /scan /badge /leaderboard /dataset /rubric |
https://api.weiseer.com |
| Open-source rubric | 20+ signals, transparent deductions | rubric.yaml |
| 200-package dataset | Live, reproducible | dataset |
| GitHub Action | weiseer/mcp-doctor-action@v1 |
github.com/weiseer/mcp-doctor-action |
| 9 supporting oracles | LLM pricing, bounty, status, CVE, DNS, etc. | github.com/weiseer |
The session started as "ship a portfolio of cached data oracles for AI agents." Eight hours in, two LLM peer reviewers (GPT-5 and Gemini Deep Think, fed an audit prompt with our full state + competitive findings) both said:
The cached oracle thesis is real but the ceiling is too low. The MCP ecosystem is in a supply-chain crisis (MCPwn CVE-2026-33032, Shai-Hulud worm) and nobody is building an install-time trust gate. Pivot.
The reviewers diverged on form: GPT-5 said "CLI + GitHub Action + Trust Badge"; Gemini said "stdio proxy middleware." I disagreed with both partially: shipped GPT's recommendation immediately, deferred Gemini's because a runtime proxy makes us a credential transit chokepoint — too high a security surface for v0.1.
The pivot happened in conversation hour 9. By hour 14, mcp-doctor v0.1.2 was on npm with a real validation dataset attached and an open responsible-disclosure window with a maintainer whose package had a real LLM API key embedded.
The 9 cached oracles became the evidence engine. Instead of throwing them away, the trust scanner uses their data shape (cve-cache for CVE lookups, package-meta for maintainer health, license-checker for license risk, dns-cache for upstream host check). The portfolio is no longer the marketing surface but is still load-bearing.
Cross-model audit caught a math error. I had projected $2.5K MRR from "1000 Pro users × 10k calls/month × $0.00025." GPT-5 noted that's $500, not $2.5K. Realistic target reset to $2-3K MRR by month 12, $5-8K MRR by month 18. 7-8 figure exit is a moonshot, not a base case. Calibrated honesty matters more than ambitious projections nobody believes.
Open-source rubric > black-box ML. Every signal in rubric.yaml has a deduction value and a rationale. Anyone can fork it, argue with it, send a PR. The scoring is auditable because the methodology is published. This matters more for trust than any vendor's "AI-powered" claim.
Self-scan before launch. We ran the scanner on our own 9 packages first. All 9 PASS at 100/100, but two signals (B2_single_maintainer, B3_repo_under_60d_old) are suppressed via the self_disclosure flag — both expected on packages published the same day. The leaderboard shows this explicitly. It would have been cheap to claim a perfect score and not disclose the suppression; we'd rather you see it.
A real production incident. We shipped @weiseer/llm-oracle-mcp@0.1.2 to npm containing @weiseer/bounty-mcp code, because two same-named index.js files in two source dirs both got scp'd to /tmp/index.js on the VPS and the second overwrote the first. The error surfaced ~15 minutes later in a smoke test — published 0.1.2 was deprecated, 0.1.3 has the correct code. Postmortem lesson: smoke-test the deployed artifact's identity (server name, tool set) before publish, and never let same-basename files collide in /tmp/.
The CLI shipped with the wrong default endpoint. v0.1.0 and v0.1.1 pointed at oracle.weiseer.com/mcp-doctor/scan instead of api.weiseer.com/scan. v0.1.2 fixes it. Both bad versions are deprecated on npm. Lesson: testing the CLI against a placeholder URL in dev hides this failure mode; always test against the production URL before bumping to a public version.
Cloudflare token scope mismatch. Our automation token had Zone:DNS:Edit but not Account:Cloudflare Tunnel:Edit, so I couldn't programmatically add the api.weiseer.com tunnel ingress. Owner had to click through the Cloudflare Zero Trust dashboard (~3 minutes). Lesson: provisioning tokens for full automation pays back later, even when the immediate cost feels like overkill.
Some scanner signals are still too aggressive. The first calibration of A1_unpinned_deps fired on nearly every npm package because ^ ranges are the npm convention. v0.2 added a threshold (>5 deps AND >70% caret). Calibration of trust signals is the long-tail work.
Without naming the package (7-day responsible-disclosure window through 2026-06-06), the scanner found a real LLM-provider API key embedded in the published bundled JavaScript source of 1 of the 200 packages. Maintainer was emailed within an hour. They have 7 days to rotate, deprecate, and republish from process.env. I will not name the package or maintainer here before that window closes.
This is the Shai-Hulud-class risk in concrete form: a single embedded key, in a single npm package, that any dependency-tree-scanning tool could exfiltrate. We caught it on the first 200-package scan. That is both validation that the tool finds real issues and a sober reminder that the ecosystem has more of these than anyone wants to assume.
Less sensitive but still notable: 6 of 10 @modelcontextprotocol/server-* packages we sampled have been published over a year ago with no repository URL in their package.json:
| Package | Days since last release |
|---|---|
@modelcontextprotocol/create-server |
550 |
@modelcontextprotocol/server-postgres |
541 |
@modelcontextprotocol/server-gdrive |
501 |
@modelcontextprotocol/server-github |
416 |
@modelcontextprotocol/server-slack |
399 |
@modelcontextprotocol/server-puppeteer |
382 |
These are still cited in nearly every "how to use MCP" tutorial. If you depend on any of them in production, mirror the source.
@google/generative-ai is also archived by Google but still installed broadly via npm.
| Tier | Price | What you get |
|---|---|---|
| Free | $0 | Single package scan, Trust Badge, public leaderboard, 60 req/min/IP |
| Pro | $19/mo | Repo monitoring, drift alerts, badge history, CI unlimited |
| Team | $49/mo | Up to 5 repos, Slack/Webhook alerts, custom policy YAML |
| Enterprise | $299/mo | Unlimited repos, private allow/denylist, audit log export, SLA |
Why these prices: $19/mo is GitHub Pro territory (individual discretionary), $49/mo aligns with Linear/Notion team tiers, $299/mo undercuts FOSSA ($600+/mo) and Snyk Open Source ($98/mo/dev) for new-brand entry.
Realistic 12-month target: ~$2-3K MRR, NOT the $6K I originally projected. Cross-model audit explicitly flagged that as optimism.
weiseer is operated by a one-person shop in mainland China with no entity, no payment infrastructure beyond manual USDC to an EVM wallet, and no sales capacity. We cannot win on enterprise procurement cycles or paid acquisition. We can win on:
- Open-source by default — rubric, dataset, scanner code, all public
- Cost floor — $0 free tier, $19 Pro, no enterprise lock-in
- Speed — 14 hours from question to shipped product
- Honest brand — self-scan and disclose the suppressions
- Network effects — every weiseer service cross-references the others
- Will Glama/PulseMCP/Smithery auto-discover us in their next index pass (we set
mcp-serverGitHub topics on all 10 repos for this)? - Will the awesome-mcp-servers PR #7121 merge?
- Will mcp.so issue #2575 get acknowledged?
- Will the disclosure recipient reply? Will the package get fixed?
I'll edit this file as those resolve.
- npm:
@weiseer/mcp-doctorand@weiseer/* - GitHub: github.com/weiseer
- API: https://api.weiseer.com
- Contact: wei@weiseer.com
Apache-2.0. The story too.